Local Citation Recommendation with Hierarchical-Attention Text Encoder and SciBERT-Based Reranking

The goal of local citation recommendation is to recommend a missing reference from the context and optionally also global context. To balance tradeoff between speed accuracy in large-scale paper database, viable approach first prefetch limited number relevant documents using efficient ranking methods then perform fine-grained reranking more sophisticated models. In that vein, BM25 has been found be tough-to-beat prefetching, which why recent work focused mainly on step. Even so, we explore prefetching with nearest neighbor search among text embeddings constructed by hierarchical attention network. When coupled SciBERT reranker fine-tuned tasks, our Attention encoder (HAtten) achieves high recall for given candidates reranked. Consequently, requires fewer rerank, yet still state-of-the-art performance various datasets such as ACL-200, FullTextPeerRead, RefSeer, arXiv.